Image sampling in diffraction grating-based display system for alignment control

ABSTRACT

A display system includes a waveguide plate comprising an in-coupling grating, an expansion grating, and a sampling grating. The display system includes a projection system configured to direct input light toward the in-coupling grating. The in-coupling grating is configured to diffract the input light to propagate within the waveguide plate. The in-coupling grating is configured to (i) cause a display portion of the input light to propagate toward the expansion grating in a manner that avoids diffraction by the expansion grating and (ii) cause a sampling portion of the input light to propagate toward the sampling grating. The expansion grating is configured to (i) diffract the display portion of the input light to cause the display portion of the input light to continue to propagate within the waveguide plate. The sampling grating is configured to diffract the sampling portion of the input light outward from the waveguide plate.

BACKGROUND

Mixed-reality (MR) systems, including virtual-reality andaugmented-reality systems, have received significant attention becauseof their ability to create truly unique experiences for their users. Forreference, conventional virtual-reality (VR) systems create a completelyimmersive experience by restricting their users' views to only a virtualenvironment. This is often achieved, in VR systems, through the use of ahead-mounted device (HMD) that completely blocks any view of the realworld. As a result, a user is entirely immersed within the virtualenvironment. In contrast, conventional augmented-reality (AR) systemscreate an augmented-reality experience by visually presenting virtualobjects that are placed in or that interact with the real world.

AR systems typically include transparent display elements through whichlight for forming images is projected for viewing by an end user. Forexample, a display element may comprise a set of transparent plates(e.g., glass, plastic, or other transparent plates) and a lightprojection system (e.g., including one or more light sources and one ormore microelectromechanical system mirrors) that projects light towardthe set of glass plates. The set of glass plates may receive and expandthe input light in multiple dimensions to provide a field of view (FOV)through which an image may be viewed by a user.

Many AR systems include separate display elements for displaying imagesto separate eyes of the user. To create a realistic immersiveexperience, the image output displayed by the separate display elementsshould be synchronized and spatially aligned. However, many factors maygive rise to inconsistencies between the image output displayed byseparate display elements within the same AR system (e.g., manufacturingvariances, temperature differences, mechanical changes over time due tovibration/shock, etc.).

Facilitating alignment between separate display elements of AR systemsis associated with many challenges. There is an ongoing need and desirefor improvements to systems and methods for facilitating image alignmentof separate display elements.

The subject matter claimed herein is not limited to embodiments thatsolve any disadvantages or that operate only in environments such asthose described above. Rather, this background is only provided toillustrate one exemplary technology area where some embodimentsdescribed herein may be practiced.

BRIEF SUMMARY

Disclosed embodiments include systems, methods, and devices for imagesampling in diffraction grating display systems for facilitatingalignment control. Although at least some examples provided herein focuson surface relief grating (SRG) displays, one will appreciate, in viewof the present disclosure, that other types of gratings may be utilized,such as holographic gratings, photonic crystals, Bragg polarizationgratings, volume gratings, and/or others.

Some disclosed display systems include a waveguide plate comprisingopposing parallel surfaces, an in-coupling grating, an expansiongrating, and a sampling grating. The display system further includes aprojection system configured to direct input light toward thein-coupling grating. The in-coupling grating is configured to diffractthe input light to cause total internal reflection of the input lightwithin the waveguide plate via the opposing parallel surfaces of thewaveguide plate. The in-coupling grating is configured to (i) cause adisplay portion of the input light to propagate within the waveguideplate toward the expansion grating and (ii) cause a sampling portion ofthe input light to propagate within the waveguide plate toward thesampling grating. The expansion grating is configured to (i) diffractthe display portion of the input light in a manner that expands afield-of-view (FOV) of the display portion of the input light in atleast a first dimension and (ii) cause the display portion of the inputlight to continue to propagate within the waveguide plate. The samplinggrating is configured to diffract the sampling portion of the inputlight outward from the waveguide plate.

Some disclosed display systems include a waveguide stack including aplurality of waveguide plates. Each waveguide plate may includerespective opposing parallel surfaces, a respective in-coupling grating,a respective expansion grating, and a respective out-coupling grating.The display system further includes a projection system configured todirect input light toward each of the respective in-coupling gratings ofthe waveguide stack. For each waveguide plate, the respectivein-coupling grating is configured to cause at least a portion of theinput light to propagate within the waveguide plate toward therespective expansion grating, and the respective expansion grating isconfigured to diffract at least the portion of the input light towardthe respective out-coupling grating to generate an expanded-FOVrepresentation of an image for viewing by an eye of a user. At least onewaveguide plate of the plurality of waveguide plates further comprises asampling grating. For the at least one waveguide plate, the respectivein-coupling grating is configured to cause a sampling portion of theinput light to propagate within the at least one waveguide plate towardthe sampling grating, and the sampling grating is configured to diffractthe sampling portion of the input light outward from the waveguideplate.

At least some disclosed methods include detecting an alignment image viaan image sensor, the alignment image at least partially capturing analignment marker depicted by a sampling portion of input light, thesampling portion of the input light being diffracted toward the imagesensor by a sampling grating of a waveguide plate, the sampling portionof the input light being diffracted toward the sampling grating by anin-coupling grating of the waveguide plate, the in-coupling grating ofthe waveguide plate being configured to receive the input light from aprojection system and (i) diffract the sampling portion of the inputlight toward the sampling grating and (ii) diffract a display portion ofthe input light toward an expansion grating of the waveguide plate forexpansion and diffraction by the expansion grating toward anout-coupling grating of the waveguide plate for further expansion anddiffraction by the out-coupling grating to generate an expanded-FOVrepresentation of at least a portion of an image depicted by the inputlight for viewing by an eye of a user. The method further includesdetecting a second alignment image via a second image sensor, the secondalignment image at least partially capturing a second alignment markerdepicted by a second sampling portion of second input light, the secondsampling portion of the second input light being diffracted toward thesecond image sensor by a second sampling grating of a second waveguideplate, the second sampling portion of the second input light beingdiffracted toward the second sampling grating by a second in-couplinggrating of the second waveguide plate, the second in-coupling grating ofthe second waveguide plate being configured to receive the second inputlight from a second projection system and (i) diffract the secondsampling portion of the second input light toward the second samplinggrating and (ii) diffract a second display portion of the second inputlight toward a second expansion grating of the second waveguide platefor expansion and diffraction by the second expansion grating toward asecond out-coupling grating of the second waveguide plate for furtherexpansion and diffraction by the second out-coupling grating to generatea second expanded-FOV representation of at least a portion of a secondimage depicted by the second input light for viewing by a second eye ofthe user. The method further includes modifying the image or the secondimage based upon the alignment image or the second alignment image.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Additional features and advantages will be set forth in the descriptionwhich follows, and in part will be obvious from the description, or maybe learned by the practice of the teachings herein. Features andadvantages of the invention may be realized and obtained by means of theinstruments and combinations particularly pointed out in the appendedclaims. Features of the present invention will become more fullyapparent from the following description and appended claims or may belearned by the practice of the invention as set forth hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features can be obtained, a more particular descriptionof the subject matter briefly described above will be rendered byreference to specific embodiments which are illustrated in the appendeddrawings. Understanding that these drawings depict only typicalembodiments and are not therefore to be considered to be limiting inscope, embodiments will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1 illustrates example components of an example system that mayinclude or be used to implement one or more disclosed embodiments;

FIGS. 2A through 2D illustrate various views of a schematicrepresentation of example components of a waveguide plate;

FIGS. 3A and 3B illustrate various views of a schematic representationof example components of a waveguide stack;

FIG. 4 illustrates a schematic representation of an example waveguideplate that includes a sampling grating, in accordance withimplementations of the present disclosure;

FIG. 5 illustrates an example k-vector diagram of light propagationthrough a waveguide plate, in accordance with implementations of thepresent disclosure; and

FIG. 6 illustrates an example flow diagram depicting acts associatedwith facilitating alignment control of image output.

DETAILED DESCRIPTION

Disclosed embodiments are generally directed to systems, methods, anddevices for image sampling in surface relief grating (SRG) displaysystems for facilitating alignment control. Although the presentdisclosure focuses, in at least some respects, on SRG display systemsand/or augmented reality (AR) systems, one will appreciate, in view ofthe present disclosure, that the principles disclosed herein are notlimited to such implementations and may be applied to other fields ofendeavor (e.g., other types of gratings and/or display devices).

Examples of Technical Benefits, Improvements, and Practical Applications

Those skilled in the art will recognize, in view of the presentdisclosure, that at least some of the disclosed embodiments may beimplemented to address various shortcomings associated with facilitatingalignment control of different SRG display systems. The followingsection outlines some example improvements and/or practical applicationsprovided by the disclosed embodiments. It will be appreciated, however,that the following are examples only and that the embodiments describedherein are in no way limited to the example improvements discussedherein.

A display system, in accordance with the present disclosure, mayimplement one or more waveguide plates that include at least anin-coupling grating, an expansion grating, and a sampling grating. Thein-coupling grating may be configured to diffract a display portion ofinput light toward the expansion grating (for further diffraction towardan out-coupling grating) and to diffract a sampling portion of the inputlight toward the sampling grating. The sampling grating may out-couplethe sampling portion of the input light toward an image sensor, whichmay capture a test image.

The input light may depict a representation of an input image. Thedisplay portion of the input light may depict portions of the inputimage intended for viewing by an eye user. The display portion of thelight diffracted by the expansion grating may propagate toward anout-coupling grating for further diffraction toward the eye of the user.Propagation of the display portion of the input light through thewaveguide plate (e.g., being diffracted by the in-coupling grating, theexpansion grating, and the out-coupling grating) may expand the displayportion of the input light to cause propagate thereof with multiplereplicas, thereby allowing the display portion of the input image to beviewable through a larger area of the out-coupling grating.

The input image may comprise an alignment marker (e.g., crosshairsarranged on a small part of the input image, such as an edge thereof),which may be depicted by the sampling portion of the input light. Thesampling grating may be arranged to receive this sampling portion of theinput light after it is diffracted by the in-coupling grating, whereas,as noted above, the expansion grating may be arranged to receive thedisplay portion of the input light after it is diffracted by thein-coupling grating. In contrast with the display portion of the inputlight, the sampling portion of the input light may avoid diffraction bythe expansion grating (e.g., even where the in-coupling grating has auniform grating period and grating orientation).

By separating the sampling portion of the input light from the displayportion of the input light, the test image may capture the alignmentmarker and may be used to facilitate alignment between the imagedisplayed via the waveguide plate and a complementary image displayed byanother waveguide plate (e.g., a separate waveguide plate arranged todisplay image content to a separate eye of a user, thereby enabling orimproving binocular viewing of image content). Because the samplingportion of the input light avoids diffraction by the expansion grating,a test image captured in accordance with the present disclosure maycomprise an improved image signal (e.g., in contrast with an approachthat includes arranging the sampling grating to receive the remainder ofthe display portion of the input light after diffraction by theexpansion grating).

Having just described some of the various high-level features andbenefits of the disclosed embodiments, attention will now be directed toFIGS. 1 through 6 . These Figures illustrate various conceptualrepresentations, architectures, methods, and supporting illustrationsrelated to the disclosed embodiments.

Example Systems and Techniques for Facilitating Alignment Control ofImage Output

Attention is now directed to FIG. 1 , which illustrates an examplesystem 100 that may include or be used to implement one or moredisclosed embodiments. FIG. 1 depicts the system 100 as a head-mounteddisplay 114 (HMD 114) configured for placement over a head of a user todisplay virtual content for viewing by the user's eyes 118A and 118B.Such an HMD 114 may comprise an augmented reality (AR) system, a virtualreality (VR) system, and/or any other type of HMD. Although the presentdisclosure focuses, in at least some respects, on a system 100implemented as an HMD 114, it should be noted that the techniquesdescribed herein may be implemented using other types ofsystems/devices, without limitation.

FIG. 1 illustrates various example components of the system 100. Forexample, FIG. 1 illustrates an implementation in which the systemincludes processor(s) 102, storage 104, sensor(s) 106, I/O system(s)108, and communication system(s) 110. Although FIG. 1 illustrates asystem 100 as including particular components, one will appreciate, inview of the present disclosure, that a system 100 may comprise anynumber of additional or alternative components.

The processor(s) 102 may comprise one or more sets of electroniccircuitries that include any number of logic units, registers, and/orcontrol units to facilitate the execution of computer-readableinstructions (e.g., instructions that form a computer program). Suchcomputer-readable instructions may be stored within storage 104. Thestorage 104 may comprise physical system memory and may be volatile,non-volatile, or some combination thereof. Furthermore, storage 104 maycomprise local storage, remote storage (e.g., accessible viacommunication system(s) 110 or otherwise), or some combination thereof.Additional details related to processors (e.g., processor(s) 102) andcomputer storage media (e.g., storage 104) will be provided hereinafter.

As will be described in more detail, the processor(s) 102 may beconfigured to execute instructions stored within storage 104 to performcertain actions. In some instances, the actions may rely at least inpart on communication system(s) 110 for receiving data from remotesystem(s) 112, which may include, for example, separate systems orcomputing devices, sensors, and/or others. The communications system(s)110 may comprise any combination of software or hardware components thatare operable to facilitate communication between on-systemcomponents/devices and/or with off-system components/devices. Forexample, the communications system(s) 110 may comprise ports, buses, orother physical connection apparatuses for communicating with otherdevices/components. Additionally, or alternatively, the communicationssystem(s) 110 may comprise systems/components operable to communicatewirelessly with external systems and/or devices through any suitablecommunication channel(s), such as, by way of non-limiting example,Bluetooth, ultra-wideband, WLAN, infrared communication, and/or others.

FIG. 1 illustrates that a system 100 may comprise or be in communicationwith sensor(s) 106. Sensor(s) 106 may comprise any device for capturingor measuring data representative of perceivable phenomenon. By way ofnon-limiting example, the sensor(s) 106 may comprise one or more imagesensors, microphones, thermometers, barometers, magnetometers,accelerometers, gyroscopes, and/or others.

Furthermore, FIG. 1 illustrates that a system 100 may comprise or be incommunication with I/O system(s) 108. I/O system(s) 108 may include anytype of input or output device such as, by way of non-limiting example,a touch screen, a mouse, a keyboard, a controller, and/or others,without limitation. For example, the I/O system(s) 108 may include adisplay system that may comprise any number of display panels, optics,laser scanning display assemblies, and/or other components.

For instance, the I/O system(s) 108 of the system 100 (e.g., implementedas an HMD 114) may comprise surface relief grating (SRG) displays 116Aand 116B configured for displaying image for viewing by eyes of a user(e.g., user eyes 118A and 118B, respectively). The SRG displays 116A and116B may each comprise one or more glass plates that include diffractiveoptical elements (DOES) disposed thereon. Although the present example(and other portions of the present disclosure) may specifically recitesurface relief gratings, other types of gratings are within the scope ofthe present disclosure.

The SRG displays 116A and 116B may be configured to receive light fromprojection systems (e.g., microelectromechanical projectors), where thelight depicts an image for viewing by the eyes 118A and 118B of theuser. The SRG displays 116A and 116B may expand a field of view (FOV) toallow the user's eyes 118A and 118B to view the image content throughlarge portions of the SRG displays 116A and 116B.

As noted above, the SRG displays 116A and 116B may each comprise one ormore respective glass plates for facilitating their image displayingfunctions. FIGS. 2A-3B depict example components of such glass plates.One will appreciate, in view of the present disclosure, that theparticular shapes/forms/configurations of the components of FIGS. 2A-3Bare provided by way of conceptual example only and are not intended tolimit the scope of the present disclosure.

FIG. 2A depicts a waveguide plate 202, which may form at least a part ofan SRG display (e.g., SRG displays 116A, 116B). FIG. 2A shows that awaveguide plate 202 may comprise various gratings, such as anin-coupling grating 204, an expansion grating 206, and an out-couplinggrating 208. The in-coupling grating 204 may be configured to receiveinput light 216 and diffract the input light 216 for propagation withinthe waveguide plate 202. The input light 216 may be generated by aprojection system 214 (e.g., a microelectromechanical projector and/orlaser and mirror system; a microdisplay panel (reflective ortransmissive) illuminated by a laser, LED, or other light source withoptics for panel illumination and for projection of the panel image,etc.) driven by a projection system driver 212. The projection systemdriver 212 may drive the projection system 214 in accordance with imageinput (e.g., image 210 of FIG. 2A) to cause the input light 216generated by the projection system 214 to depict or represent the image210.

After diffraction by the in-coupling grating 204, the input light 216 isfurther diffracted by the expansion grating 206 and the out-couplinggrating 208. The out-coupling grating 208 may diffract an expandedrepresentation 218 of at least a portion of the image 210 outward fromthe waveguide plate 202 for viewing by an eye of a user 220. Diffractionof the input light 216 through the various gratings of the waveguideplate 202 causes replica expansion of the input light 216 which maycause the image pixels forming the image 210 to be visible throughmultiple locations on the out-coupling grating 208 (e.g., with the imageappearing to be at an infinite distance and observable across a range ofdifferent eye positions).

FIGS. 2B, 2C, and 2D provide additional insight into propagation of theinput light 216 through the waveguide plate 202. For instance, FIG. 2Bshows a top view of the waveguide plate 202, FIG. 2C shows a frontsectional view of the waveguide plate 202 (sectioned along dashed line(A) shown in FIG. 2B), and FIG. 2D shows a side sectional view of thewaveguide plate 202 (sectioned along dashed line (B) shown in FIG. 2B).Relevant x, y, and z-axes are illustrated in FIGS. 2B through 2D forclarity.

FIG. 2B shows the input light 216 diffracted from the in-couplinggrating 204 toward the expansion grating 206 of the waveguide plate 202.FIG. 2C furthermore shows that the in-coupling grating 204 diffracts theinput light 216 in a manner that causes total internal reflection of theinput light 216 within the waveguide plate 202 (e.g., between opposingparallel surfaces 222 and 224 of the waveguide plate 202). The inputlight 216 propagates to various portions of the expansion grating 206.The expansion grating 206 thus allows the input light 216 to expand inone direction/dimension (e.g., along the length of the expansion grating206).

As shown in FIG. 2D, the expansion grating 206 diffracts the input light216 in a manner that causes the input light 216 to continue to propagatewithin the waveguide plate 202 (e.g., via total internal reflection)toward various portions of the out-coupling grating 208, allowing theinput light to further expand in another direction/dimension (e.g.,along the length of the out-coupling grating 208). In this regard, theout-coupling grating 208 may also facilitate expansion of the inputlight 216 within the waveguide plate 202.

The out-coupling grating 208 is configured to diffract the input light216 outward from the waveguide plate 202, shown in FIG. 2D by theexpanded representation 218 diffracting outward from the waveguide plate202. As noted above, the expanded representation 218 may be viewed by aneye of a user from various viewing angles.

While FIGS. 2A through 2D illustrate propagation of a single beam ofinput light through the waveguide plate 202 (e.g., forming a pixel of animage), multiple beams of input light may propagate through thewaveguide plate 202 (e.g., being projected toward the in-couplinggrating 404 with different incident angles) to form a representation ofan image (e.g., to form multiple pixels of an input image, such as image210 of FIG. 2A).

Furthermore, in some instances, multiple waveguide plates are utilizedto form a representation of an input image. For example, FIGS. 3A and 3Billustrate various views of a waveguide stack 300 that includes aplurality of waveguide plates 302A, 302B, and 302C usable to expand aprojection of input light 310 representing an image for viewing by auser.

As shown in FIGS. 3A and 3B, each of the waveguide plates 302A, 302B,and 302C includes a respective in-coupling grating 304A, 304B, and 304C,a respective expansion grating 306A, 306B, and 306B, and a respectiveout-coupling grating 308A, 308B, and 308C. As shown in FIG. 3A,different portions of the input light 310 are in-coupled by thedifferent in-coupling gratings 304A, 304B, and 304C. For instance,portion 310A of the input light 310 is in-coupled by in-coupling grating304A for propagation through waveguide plate 302A toward expansiongrating 306A. Similarly, portion 310B of the input light 310 isin-coupled by in-coupling grating 304B for propagation through waveguideplate 302B toward expansion grating 306B. Furthermore, portion 310C ofthe input light 310 is in-coupled by in-coupling grating 304C forpropagation through waveguide plate 302C toward expansion grating 306C.The different portions 310A, 310B, and 310C of the input light maycorrespond to different color channels (e.g., red, green, blue) and/ordifferent image regions.

As shown in FIG. 3B, expansion grating 306A diffracts portion 310A ofthe input light 310 toward out-coupling grating 308A to causeout-coupling of expanded representation 312A. Similarly, expansiongrating 306B diffracts portion 310B of the input light 310 towardout-coupling grating 308B to cause out-coupling of expandedrepresentation 312B. Furthermore, expansion grating 306C diffractsportion 310C of the input light 310 toward out-coupling grating 308C tocause out-coupling of expanded representation 312C. The differentexpanded portions 312A, 312B, and 312C may be viewed simultaneously by auser and perceived as a depiction of an input image.

As noted above, in some instances, a waveguide plate (or waveguidestack) may be utilized in combination with another waveguide plate (orwaveguide stack) to provide users with binocular representations ofinput imagery (e.g., see HMD 114 of FIG. 1 , with different SRG displays116A and 116B for presentation to different user eyes 118A and 118B). Inaccordance with the present disclosure, alignment components may beimplemented to ensure that the imagery shown via the different waveguideplates (or different waveguide stacks) are aligned to provide arealistic representation of the input imagery.

FIG. 4 illustrates a schematic representation of an example waveguideplate 402 that includes a sampling grating 410 in addition to anin-coupling grating 404, an expansion grating 406, and an out-couplinggrating 408. The in-coupling grating 404, the expansion grating 406, andthe out-coupling grating 408 correspond generally to commonly namedgratings described hereinabove.

In some instances, the sampling grating 410 is disposed on the same sideof the waveguide plate 402 as the in-coupling grating 404, the expansiongrating 406, and/or the out-coupling grating. In some instances, thesampling grating 410 has the same grating period and/or gratingorientation as the in-coupling grating 404. As will be described in moredetail hereinbelow, the sampling grating 410 diffracts light outwardfrom the waveguide plate 402. However, the sampling grating 410 isdistinct from the out-coupling grating 408 and may have a differentgrating period and/or grating orientation than the out-coupling grating408.

As shown in the example depicted in FIG. 4 , the in-coupling grating 404is configured to in-couple input light 412 to the waveguide plate 402for propagation therethrough. In the example of FIG. 4 , the input light412 comprises a display portion 416 and a sampling portion. Thein-coupling grating 404 causes the display portion 416 of the inputlight to propagate within the waveguide plate 402 toward the expansiongrating 406 for diffraction by the expansion grating 406 toward theout-coupling grating 408. The in-coupling grating 404 causes thesampling portion 414 to propagate within the waveguide plate 402 in amanner that avoids diffraction by the expansion grating 406, allowingthe sampling portion 414 to reach the sampling grating 410 without beingdiminished in intensity by the expansion grating 406. For example, theexpansion grating 406 and the sampling grating 410 may be positioned onthe waveguide plate 402 to receive different portions of the input light412 diffracted by the in-coupling grating 404 (e.g., even where thein-coupling grating 404 comprises a single grating where the region ofthe in-coupling grating 404 that diffracts the display portion 416 ofthe input light 412 toward the expansion grating 406 has the samegrating period and grating orientation as the region of the in-couplinggrating 404 that diffracts the sampling portion 414 of the input light412 toward the sampling grating 410).

As indicated above, the sampling grating 410 diffracts the samplingportion 414 of the input light 412 outward from the waveguide plate 402,such as toward an image sensor (e.g., of sensors 106 of a system 100 orHMD 114) to facilitate detection of an alignment image by the imagesensor. As also indicated above, the sampling portion 414 of the inputlight may depict a portion of an input image (e.g., image 210) thatincludes an alignment marker (e.g., crosshairs or other symbols, icons,features, etc.). The alignment image may be analyzed to detectdistortion and/or undesired transformation of the alignment marker, andcompensatory transformations/modifications may be applied to subsequentimagery to correct for the detected image distortions.

In some instances, such alignment functionality may be advantageous whena waveguide plate 402 configured to display imagery to one user eyeoperates in conjunction with another waveguide plate configured todisplay corresponding imagery to another user eye (e.g., to providebinocular views of input imagery). For instance, the SRG displays 116Aand 116B of the HMD 114 of FIG. 1 may each include respective sets ofcomponents shown in FIG. 4 . Each may receive input light from aprojector system and in-couple the light via an in-coupling grating. Foreach SRG display 116A and 116B, the in-coupling grating may diffract asampling portion of the input light toward a sampling grating forout-coupling toward an image sensor. The image sensors may capture thesampling portion (which may depict an alignment marker), andtransformations/modifications may be applied to subsequent imagery toprovide spatially accurate and/or undistorted views of input imagery(e.g., binocular imagery).

FIG. 5 illustrates an example k-vector diagram of light propagationthrough a waveguide plate that includes an in-coupling grating,expansion grating, out-coupling grating, and sampling grating, inaccordance with the present disclosure. The coordinate system of thek-vector representation of FIG. 5 is selected to have plate surfaces inx-y-plane and z axis normal to the plate surfaces. Each point in diagramcorresponds to a projection of vector

$\frac{k}{k_{0}}$

in the x-y-plane,

${\left( \frac{k_{x}}{k_{0}} \right)i} + {\left( \frac{k_{y}}{k_{0}} \right){j.}}$

Here, k represents the wave vector of light,

${k_{0} = \frac{2\pi}{\lambda_{0}}},$

where λ₀ represents vacuum wavelength. The direction of the vector

$\frac{k}{k_{0}}$

is the direction or light propagation, and the magnitude of the vector

$\frac{k}{k_{0}}$

is the retractive index n of the material in which the light ispropagating. The inner circle in the diagram of FIG. 5 has a radius of 1and corresponds to light rays that can propagate in air and can betransmitted through surfaces of waveguide plates as discussed herein.The external boundary of the diagram of FIG. 5 has a radiuscorresponding to the refractive index of the plate material (typicallyglass, with a refractive index of 1.7 in the example shown in FIG. 5 ).Light located in the (donut-shaped) zone of the diagram between theexternal boundary (refractive index of 1.7) and the inner circle withthe radius of 1 may be regarded as propagating inside the plate,confined therein by total internal reflection. The x-y-projection vectorin the diagram of FIG. 5 (from the diagram origin (0, 0) to the point(k_(x), k_(y)) shows the direction of light propagation in the waveguideplate.

The region 502 shown by a solid line in FIG. 5 may represent theboundary inside which the image light projected into the waveguide plateis confined. Each pixel of the image corresponds to a certain directionof light represented by a point inside the region 502. When light isdiffracted by the in-coupling grating (e.g., in-coupling grating 404),the light is translated in the k-vector presentation by the gratingvector of the in-coupling grating. Region 504 in FIG. 5 represents thelight after diffraction by the in-coupling grating. In some instances,as shown by the example of FIG. 5 , part of the input image light maybecome directed outside the external boundary (e.g., the outer boundarywith a radius of 1.7 in the diagram of FIG. 5 ). Light cannot propagatein the waveguide plate in such locations of the diagram, so this thispart of image is not diffracted by the in-coupling grating. The diagramof FIG. 5 shows that the region 504 is cut by the external boundary(with a radius of 1.7).

As noted above, the in-coupled light propagates through the expansiongrating (e.g., expansion grating 406) and is diffracted by the expansiongrating toward the out-coupling grating (e.g., out-coupling grating408). The light after diffraction by the expansion grating isrepresented in FIG. 5 by region 506. As shown in FIG. 5 , region 506 isfurther cut by the external boundary (with a radius of 1.7). The areacut from the internal area of region 504 cannot be diffracted by theexpansion grating, and thus it remains in the original direction andtravels through the expansion grating undeflected. The dashed lineinside of region 504 (labelled region 508 in FIG. 5 ) indicates theboundary between light diffracted by the expansion grating (e.g.,corresponding to the display portion 416 of the input light 412 as shownand described with reference to FIG. 4) and not diffracted by theexpansion grating (e.g., corresponding to the sampling portion 414 ofthe input light 412 as shown and described with reference to FIG. 4 ).

As noted above, the out-coupling grating diffracts the light outwardfrom the waveguide plate, depicted in the diagram of FIG. 5 by atranslation from region 506 back to region 502. As part of the image isnot diffracted fully through the waveguide plate, the field-of-viewtransmitted may be limited. This is shown by the dashed line insideregion 502, representing the boundary of the output image.

In accordance with the present disclosure, the part of the image contentnot diffracted by the expansion grating (e.g., represented by region 508and corresponding to the sampling portion 414) may be used for samplingvia sampling grating (e.g., the sampling grating may facilitate atranslation in k-vector space from the region 508 back to the region502). Such a configuration may facilitate a higher intensity of thesampled image. Also, intensity variation in the sampled image (which mayresult from manufacturing tolerances associated with the expansiongrating) may be avoided.

In some instances, the sampled region (e.g., region 508 in FIG. 5 ) maycorrespond to the part of FOV outside the actual image of the display.In some instances, the sampled region may include image content, andsuch image content may be transmitted to the output grating throughdifferent in-coupling and expansion gratings on the waveguide plate(and/or a different waveguide plate within a waveguide stack).

The principles discussed hereinabove with reference to FIGS. 4 and 5 maybe applied to waveguide stacks comprising multiple waveguide plates(e.g., similar to the examples shown in FIGS. 3A and 3B). For example,each waveguide plate of a waveguide stack may include a respectivein-coupling grating, a respective expansion grating, and a respectiveout-coupling grating. A projection system may be configured to directinput light toward each of the respective in-coupling gratings of thewaveguide stack.

For each waveguide plate, the respective in-coupling grating may beconfigured to cause at least a portion of the input light to propagatewithin the waveguide plate toward the respective expansion grating. Therespective expansion grating may be configured to diffract at least theportion of the input light toward the respective out-coupling grating togenerate an expanded-FOV representation of an image for viewing by aneye of a user.

At least one waveguide plate of the plurality of waveguide platesfurther comprises a sampling grating. For at least this waveguide plate,the respective in-coupling grating is configured to cause a samplingportion of the input light to propagate within the waveguide platetoward the sampling grating. The sampling grating of this waveguide isconfigured to diffract the sampling portion of the input light outwardfrom the waveguide plate.

In some instances, each waveguide plate of the waveguide stack includesa respective sampling grating configured to receive a respectivesampling portion of the input light diffracted by the respectivein-coupling grating and diffract the respective sampling portion outwardfrom the waveguide stack. In some instances, each of the waveguideplates of the waveguide stack is associated with a respective colorchannel. The input light may transmit through one or more of thewaveguide plates of the waveguide stack to reach the respectivein-coupling grating of one or more other waveguide plates of thewaveguide stack.

Example Method(s) for Facilitating Alignment Control of Image Output

The following discussion now refers to a number of methods and methodacts that may be performed by the disclosed systems. Although the methodacts are discussed in a certain order and illustrated in a flow chart asoccurring in a particular order, no particular ordering is requiredunless specifically stated, or required because an act is dependent onanother act being completed prior to the act being performed. One willappreciate that certain embodiments of the present disclosure may omitone or more of the acts described herein.

FIG. 6 illustrates an example flow diagram 600 depicting acts associatedwith facilitating alignment control of image output. The discussion ofthe various acts represented in the flow diagrams include references tovarious hardware components described in more detail with reference toFIG. 1 .

Act 602 of flow diagram 600 includes detecting an alignment image via animage sensor, the alignment image at least partially capturing analignment marker depicted by a sampling portion of input light, thesampling portion of the input light being diffracted toward the imagesensor by a sampling grating of a waveguide plate, the sampling portionof the input light being diffracted toward the sampling grating by anin-coupling grating of the waveguide plate, the in-coupling grating ofthe waveguide plate being configured to receive the input light from aprojection system and (i) diffract the sampling portion of the inputlight toward the sampling grating and (ii) diffract a display portion ofthe input light toward an expansion grating of the waveguide plate forexpansion and diffraction by the expansion grating toward anout-coupling grating of the waveguide plate for further expansion anddiffraction by the out-coupling grating to generate an expanded-FOVrepresentation of at least a portion of an image depicted by the inputlight for viewing by an eye of a user. Act 602 is performed, in someinstances, by a system utilizing processor(s) 102, storage 104,sensor(s) 106, input/output system(s) 108, communication system(s) 110,and/or other components.

Act 604 of flow diagram 600 includes detecting a second alignment imagevia a second image sensor, the second alignment image at least partiallycapturing a second alignment marker depicted by a second samplingportion of second input light, the second sampling portion of the secondinput light being diffracted toward the second image sensor by a secondsampling grating of a second waveguide plate, the second samplingportion of the second input light being diffracted toward the secondsampling grating by a second in-coupling grating of the second waveguideplate, the second in-coupling grating of the second waveguide platebeing configured to receive the second input light from a secondprojection system and (i) diffract the second sampling portion of thesecond input light toward the second sampling grating and (ii) diffracta second display portion of the second input light toward a secondexpansion grating of the second waveguide plate for expansion anddiffraction by the second expansion grating toward a second out-couplinggrating of the second waveguide plate for further expansion anddiffraction by the second out-coupling grating to generate a secondexpanded-FOV representation of at least a portion of a second imagedepicted by the second input light for viewing by a second eye of theuser. Act 604 is performed, in some instances, by a system utilizingprocessor(s) 102, storage 104, sensor(s) 106, input/output system(s)108, communication system(s) 110, and/or other components.

Act 606 of flow diagram 600 includes modifying the image or the secondimage based upon the alignment image or the second alignment image. Act606 is performed, in some instances, by a system utilizing processor(s)102, storage 104, sensor(s) 106, input/output system(s) 108,communication system(s) 110, and/or other components.

Additional Details Related to Computer Systems

Disclosed embodiments may comprise or utilize a special purpose orgeneral-purpose computer including computer hardware, as discussed ingreater detail below. Disclosed embodiments also include physical andother computer-readable media for carrying or storingcomputer-executable instructions and/or data structures. Suchcomputer-readable media can be any available media that can be accessedby a general-purpose or special-purpose computer system.Computer-readable media that store computer-executable instructions inthe form of data are one or more “physical computer storage media” or“hardware storage device(s).” Computer-readable media that merely carrycomputer-executable instructions without storing the computer-executableinstructions are “transmission media.” Thus, by way of example and notlimitation, the current embodiments can comprise at least two distinctlydifferent kinds of computer-readable media: computer storage media andtransmission media.

Computer storage media (aka “hardware storage device”) arecomputer-readable hardware storage devices, such as RAM, ROM, EEPROM,CD-ROM, solid state drives (“SSD”) that are based on RAM, Flash memory,phase-change memory (“PCM”), or other types of memory, or other opticaldisk storage, magnetic disk storage or other magnetic storage devices,or any other medium that can be used to store desired program code meansin hardware in the form of computer-executable instructions, data, ordata structures and that can be accessed by a general-purpose orspecial-purpose computer.

A “network” is defined as one or more data links that enable thetransport of electronic data between computer systems and/or modulesand/or other electronic devices. When information is transferred orprovided over a network or another communications connection (eitherhardwired, wireless, or a combination of hardwired or wireless) to acomputer, the computer properly views the connection as a transmissionmedium. Transmission media can include a network and/or data links whichcan be used to carry program code in the form of computer-executableinstructions or data structures and which can be accessed by a generalpurpose or special purpose computer. Combinations of the above are alsoincluded within the scope of computer-readable media.

Further, upon reaching various computer system components, program codemeans in the form of computer-executable instructions or data structurescan be transferred automatically from transmission computer-readablemedia to physical computer-readable storage media (or vice versa). Forexample, computer-executable instructions or data structures receivedover a network or data link can be buffered in RAM within a networkinterface module (e.g., a “NIC”), and then eventually transferred tocomputer system RAM and/or to less volatile computer-readable physicalstorage media at a computer system. Thus, computer-readable physicalstorage media can be included in computer system components that also(or even primarily) utilize transmission media.

Computer-executable instructions comprise, for example, instructions anddata which cause a general-purpose computer, special purpose computer,or special purpose processing device to perform a certain function orgroup of functions. The computer-executable instructions may be, forexample, binaries, intermediate format instructions such as assemblylanguage, or even source code. Although the subject matter has beendescribed in language specific to structural features and/ormethodological acts, it is to be understood that the subject matterdefined in the appended claims is not necessarily limited to thedescribed features or acts described above. Rather, the describedfeatures and acts are disclosed as example forms of implementing theclaims.

Disclosed embodiments may comprise or utilize cloud computing. A cloudmodel can be composed of various characteristics (e.g., on-demandself-service, broad network access, resource pooling, rapid elasticity,measured service, etc.), service models (e.g., Software as a Service(“SaaS”), Platform as a Service (“PaaS”), Infrastructure as a Service(“IaaS”), and deployment models (e.g., private cloud, community cloud,public cloud, hybrid cloud, etc.).

Those skilled in the art will appreciate that the invention may bepracticed in network computing environments with many types of computersystem configurations, including, personal computers, desktop computers,laptop computers, message processors, hand-held devices, multi-processorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, pagers, routers, switches, wearable devices, and the like. Theinvention may also be practiced in distributed system environments wheremultiple computer systems (e.g., local and remote systems), which arelinked through a network (either by hardwired data links, wireless datalinks, or by a combination of hardwired and wireless data links),perform tasks. In a distributed system environment, program modules maybe located in local and/or remote memory storage devices.

Alternatively, or in addition, the functionality described herein can beperformed, at least in part, by one or more hardware logic components.For example, and without limitation, illustrative types of hardwarelogic components that can be used include Field-programmable Gate Arrays(FPGAs), Program-specific Integrated Circuits (ASICs),Application-specific Standard Products (ASSPs), System-on-a-chip systems(SOCs), Complex Programmable Logic Devices (CPLDs), central processingunits (CPUs), graphics processing units (GPUs), and/or others.

As used herein, the terms “executable module,” “executable component,”“component,” “module,” or “engine” can refer to hardware processingunits or to software objects, routines, or methods that may be executedon one or more computer systems. The different components, modules,engines, and services described herein may be implemented as objects orprocessors that execute on one or more computer systems (e.g., asseparate threads).

One will also appreciate how any feature or operation disclosed hereinmay be combined with any one or combination of the other features andoperations disclosed herein. Additionally, the content or feature in anyone of the figures may be combined or used in connection with anycontent or feature used in any of the other figures. In this regard, thecontent disclosed in any one figure is not mutually exclusive andinstead may be combinable with the content from any of the otherfigures.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or characteristics. The described embodimentsare to be considered in all respects only as illustrative and notrestrictive. The scope of the invention is, therefore, indicated by theappended claims rather than by the foregoing description. All changeswhich come within the meaning and range of equivalency of the claims areto be embraced within their scope.

We claim:
 1. A display system for facilitating alignment control ofimage output, the display system comprising: a waveguide platecomprising opposing parallel surfaces, the waveguide plate furthercomprising an in-coupling grating, an expansion grating, and a samplinggrating; and a projection system configured to direct input light towardthe in-coupling grating, the in-coupling grating being configured todiffract the input light to cause total internal reflection of the inputlight within the waveguide plate via the opposing parallel surfaces ofthe waveguide plate, wherein: the in-coupling grating is configured to(i) cause a display portion of the input light to propagate within thewaveguide plate toward the expansion grating and (ii) cause a samplingportion of the input light to propagate within the waveguide platetoward the sampling grating in a manner that causes the sampling portionof the input light to avoid diffraction by the expansion grating, theexpansion grating is configured to (i) diffract the display portion ofthe input light in a manner that expands a field-of-view (FOV) of thedisplay portion of the input light in at least a first dimension and(ii) cause the display portion of the input light to continue topropagate within the waveguide plate, and the sampling grating isconfigured to diffract the sampling portion of the input light outwardfrom the waveguide plate.
 2. The display system of claim 1, wherein thein-coupling grating, the expansion grating, or the sampling gratingcomprises a surface relief grating (SRG).
 3. The display system of claim1, wherein a region of the in-coupling grating that diffracts thedisplay portion of the input light toward the expansion gratingcomprises a same grating period and grating orientation as a region ofthe in-coupling grating that diffracts the sampling portion of the inputlight toward the sampling grating.
 4. The display system of claim 1,wherein the sampling grating and the expansion grating are disposed on asame parallel surface of the waveguide plate.
 5. The display system ofclaim 1, wherein the sampling grating comprises a same grating period orgrating orientation as the in-coupling grating.
 6. The display system ofclaim 1, wherein the waveguide plate further comprises an out-couplinggrating configured to diffract the display portion of the input lightexpanded by the expansion grating outward from the waveguide plate. 7.The display system of claim 6, wherein the out-coupling grating isconfigured to further expand the display portion of the input lightexpanded by the expansion grating in at least a second dimension.
 8. Thedisplay system of claim 6, wherein the sampling grating is distinct fromthe out-coupling grating.
 9. The display system of claim 6, wherein thesampling grating comprising a different grating period or gratingorientation than the out-coupling grating.
 10. The display system ofclaim 6, further comprising an image sensor configured to receive thesampling portion of the input light diffracted outward from thewaveguide plate by the sampling grating.
 11. The display system of claim10, wherein: the out-coupling grating of the waveguide plate isconfigured to direct an expanded-FOV representation of at least aportion of an image depicted by the input light for viewing by a firsteye of a user, and the display system further comprises: a secondwaveguide plate comprising a second in-coupling grating, a secondexpansion grating, a second sampling grating, and a second out-couplinggrating; a second image sensor; and a second projection systemconfigured to direct second input light toward the second in-couplinggrating, the second in-coupling grating being configured to (i) diffracta second display portion of the second input light toward the secondexpansion grating for expansion and diffraction of the second displayportion by the second expansion grating toward the second out-couplinggrating for further expansion and diffraction by the second out-couplinggrating to generate a second expanded-FOV representation of at least aportion of a second image depicted by the second input light for viewingby a second eye of the user, and (ii) diffract a second sampling portionof the second input light toward the second sampling grating forout-coupling toward the second image sensor.
 12. The display system ofclaim 11, wherein: the image depicted by the input light comprises analignment marker, the sampling portion of the input light depicting thealignment marker, the second image depicted by the second input lightcomprises a second alignment marker, the second sampling portion of thesecond input light depicting the second alignment marker, and thedisplay system further comprises: one or more processors; and one ormore hardware storage devices that store instructions that areexecutable by the one or more processors to configure the display systemto: detect an alignment image via the image sensor, the alignment imageat least partially capturing the alignment marker; detect a secondalignment image via the second image sensor, the second alignment imageat least partially capturing the second alignment marker; and modify theimage or the second image based upon the alignment image or the secondalignment image or based upon a comparison of the alignment image andthe second alignment image.
 13. A display system for facilitatingalignment control of image output, the display system comprising: awaveguide stack comprising a plurality of waveguide plates, eachwaveguide plate comprising respective opposing parallel surfaces, eachwaveguide plate further comprising a respective in-coupling grating, arespective expansion grating, and a respective out-coupling grating; anda projection system configured to direct input light toward each of therespective in-coupling gratings of the waveguide stack, wherein: foreach waveguide plate, the respective in-coupling grating is configuredto cause at least a portion of the input light to propagate within thewaveguide plate toward the respective expansion grating, and therespective expansion grating is configured to diffract at least theportion of the input light toward the respective out-coupling grating togenerate an expanded-FOV representation of an image for viewing by aneye of a user, at least one waveguide plate of the plurality ofwaveguide plates further comprises a sampling grating, and for the atleast one waveguide plate, the respective in-coupling grating isconfigured to cause a sampling portion of the input light to propagatewithin the at least one waveguide plate toward the sampling grating in amanner that causes the sampling portion of the input light to avoiddiffraction by the respective expansion grating, and the samplinggrating is configured to diffract the sampling portion of the inputlight outward from the waveguide plate.
 14. The display system of claim13, wherein one or more gratings of the waveguide stack comprise surfacerelief gratings (SRGs).
 15. The display system of claim 13, wherein, forthe at least one waveguide plate, a region of the respective in-couplinggrating that diffracts at least the portion of the input light towardthe expansion grating comprises a same grating period and gratingorientation as a region of the respective in-coupling grating thatdiffracts the sampling portion of the input light toward the samplinggrating.
 16. The display system of claim 13, wherein each waveguideplate of the waveguide stack comprises a respective sampling gratingconfigured to receive a respective sampling portion of the input lightdiffracted by the respective in-coupling grating and diffract therespective sampling portion outward from the waveguide stack.
 17. Thedisplay system of claim 13, wherein each of the waveguide plates of thewaveguide stack is associated with a respective color channel, andwherein the input light transmits through one or more of the waveguideplates of the waveguide stack to reach the respective in-couplinggrating of one or more other waveguide plates of the waveguide stack.18. The display system of claim 13, wherein, for the at least onewaveguide plate, the sampling grating comprises a same grating period orgrating orientation as the respective in-coupling grating.
 19. Thedisplay system of claim 13, wherein the sampling grating comprises adifferent grating period or grating orientation than the respectiveout-coupling grating.
 20. A method for facilitating alignment control ofimage output, the method comprising: detecting an alignment image via animage sensor, the alignment image at least partially capturing analignment marker depicted by a sampling portion of input light, thesampling portion of the input light being diffracted toward the imagesensor by a sampling grating of a waveguide plate, the sampling portionof the input light being diffracted toward the sampling grating by anin-coupling grating of the waveguide plate, the in-coupling grating ofthe waveguide plate being configured to receive the input light from aprojection system and (i) diffract a display portion of the input lighttoward an expansion grating of the waveguide plate for expansion anddiffraction by the expansion grating toward an out-coupling grating ofthe waveguide plate for further expansion and diffraction by theout-coupling grating to generate an expanded-FOV representation of atleast a portion of an image depicted by the input light for viewing byan eye of a user and (ii) diffract the sampling portion of the inputlight toward the sampling grating in a manner that causes the samplingportion of the input light to avoid diffraction by the expansiongrating; detecting a second alignment image via a second image sensor,the second alignment image at least partially capturing a secondalignment marker depicted by a second sampling portion of second inputlight, the second sampling portion of the second input light beingdiffracted toward the second image sensor by a second sampling gratingof a second waveguide plate, the second sampling portion of the secondinput light being diffracted toward the second sampling grating by asecond in-coupling grating of the second waveguide plate, the secondin-coupling grating of the second waveguide plate being configured toreceive the second input light from a second projection system and (i)diffract a second display portion of the second input light toward asecond expansion grating of the second waveguide plate for expansion anddiffraction by the second expansion grating toward a second out-couplinggrating of the second waveguide plate for further expansion anddiffraction by the second out-coupling grating to generate a secondexpanded-FOV representation of at least a portion of a second imagedepicted by the second input light for viewing by a second eye of theuser and (ii) diffract the second sampling portion of the second inputlight toward the second sampling grating in a manner that causes thesecond sampling portion of the second input light to avoid diffractionby the second expansion grating; and modifying the image or the secondimage based upon the alignment image or the second alignment image orbased upon a comparison of the alignment image and the second alignmentimage.